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TOWARDS  A  PHYSIOLOGICALLY  BASED  HUD  SYMBOLOGY 


INTRODUCTION 

Perhaps  the  most  important  piece  of  avionics  in  the  modern  fighter  aircraft 
is  the  head-up  display  (HUD).  It  also  remains  one  of  the  most  controversial  of 
all  cockpit  instruments,  as  evidenced  by  a  recent  series  of  papers  (1-4)  and  a 
recent  symposium  workshop  (*).  Despite  its  important  role  in  allowing  the 
pilot  to  focus  on  the  out-the-window  (OTW)  environment,  the  current  HUD  clearly 
possesses  major  disadvantages  in  terms  of  the  quality  of  its  information 
presentation  (especially  in  its  attitude  symbology),  its  compatibility  with 
head-down  displays,  and  its  training  costs.  Should  the  HUD  become  a  primary 
flight  display  as  envisioned,  it  is  doubtful  whether  current  symbologies  or 
even  slightly  modified  ones  will  prove  capable  of  serving  the  needs  of  a 
primary  flight  director  and  the  needs  of  an  OTW  target-acquisition  and  weapons  - 
delivery  system.  Accordingly,  the  intent  of  this  paper  is  to  stimulate  new  HUD 
design  concepts  based  on  an  understanding  of  the  physiological  mechanisms  and 
the  ecological  origins  of  visual  attention  and  perception  in  humans.  Given  the 
remarkable  three-dimensional  capabilities  of  low-cost  computer  graphics  sys¬ 
tems,  it  is  clear  that  the  greatest  limitation  in  the  development  of  a  new  HUD 
symbology  is  no  longer  computational  capability,  but  rather  conceptual  crea¬ 
tivity. 

The  essential  requirement  of  future  HUDs  will  be  to  improve  their  delivery 
of  primary  flight  information  while  retaining  a  "see-through"  quality.  The 
primary  obstacles  to  this  goal  are  a)  limited  total  display  area,  b)  excess 
cluttering  of  the  limited  space  available,  preventing  an  adequate  "see-through" 
capability  and  OTW  attentional  focus,  and  c)  reliance  on  a  monochrome  display, 
limiting  the  amount  of  potential  cueing  available  to  the  pilot.  Given  that  a) 
and  probably  c)  will  remain  in  the  HUD  displays  of  the  immediate  future,  the 
means  of  avoiding  further  clutter  to  an  already  saturated  display  may  require  a 
break  with  previous  symbology  concepts  towards  a  realignment  that  efficiently 
exploits  the  natural  perceptual  strengths  of  the  human  visual  system.  In  par¬ 
ticular,  the  ideal  HUD  symbology  must  a)  be  consistent  with  the  three-dimen¬ 
sional  structure  of  human  visual  attention,  so  that  HUD  space  can  be  properly 
prioritized  to  direct  pilots'  attentional  resources  toward  "far"  vision  to  the 
maximum  extent  possible;  b)  exploit  the  global  perceptual  capabilities  of  the 
human  visual  system  to  spare  valuable  focal  attentional  resources  yet  maintain 
the  "see-through"  aspect  of  the  HUD;  c)  create  an  ecologically  valid  symbology 
which  effectively  simulates  those  preattentive  visual  cues  which  are  used  in 
figure-ground  segregation  during  everyday  locomotion  and  object  scanning;  and 
d)  apply  the  proper  frame  of  reference  of  the  movement  of  the  aircraft  in  rela¬ 
tion  to  the  visual  horizon. 

The  following  sections  will  briefly  outline  various  aspects  of  human  visual 
attention  and  perception  as  they  pertain  to  the  above  issues.  Prototype 
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displays  based  on  an  understanding  of  the  physiological  and  ecological  basis  of 
visual  perception  will  illustrate  the  potential  of  a  physiologically  based 
symbology  in  accomplishing  the  goals  of  the  future  HUD. 


THE  STRUCTURE  OF  VISUAL  ATTENTION 

During  the  past  two  decades,  it  has  been  established  tnat  visual  attention 
consists  of  two  basic  forms:  spatial  attention  (5)  and  object  attention  (6). 
Spatial  attention  refers  to  the  manner  in  which  we  atteno  to  different  regions 
of  visual  space,  while  object  attention  refers  to  the  manner  in  which  we  attend 
to  different  features  of  an  object  that  distinguish  it  from  the  visual  back¬ 
ground  or  other  objects. 

In  the  frontal  plane,  the  visual  quadrant  represents  the  key  module  in  the 
structure  of  spatial  attention.  For  example,  it  has  been  shown  that  the  "spot¬ 
light"  of  visual  attention  diminishes  markedly  when  one  crosses  the  horizontal 
and  vertical  meridians  (7).  Thus,  ttie  first  rule  in  maximizing  two-dimensional 
visual  display  efficiency  is  to  arrange  information  in  a  quadrant  format,  so 
that  all  information  in  a  particular  quadrant  can  be  taken  in  during  a  single 
"attentional  glance".  This  concept  requires  that  most  if  not  all  information 
in  that  quadrant  should  be  related  to  the  same  basic  function  (e.g.,  altitude 
i nformation ) . 

Recent  findings  suggest  that  spatial  attention  may  also  possess  a  third 
dimension,  involving  "near"  and  "far"  visual  space.  For  instance,  when  we 
attend  to  far  visual  space  (as  pilots  typically  do),  information  in  near  vision 
is  less  easily  perceived  (8).  The  distinction  between  "far"  vs.  "near"  visual 
attention  should  not  be  confused  with  "far"  vs.  "near"  optical  focus,  although 
the  two  are  normally  related  ( i . e . ,  we  usually  attend  to  that  region  of  space 
upon  which  our  eyes  are  focused).  This  distinction  is  important  because  we  can 
also  focus  on  one  region  of  space  while  attending  to  another  (e.g.,  fixating  on 
an  object  while  attending  to  the  motion  of  our  hand  reaching  for  it). 

The  division  of  attention  into  "far"  and  "near"  space  appears  to  have 
evolved  in  primates  (9),  and  may  not  be  independent  of  the  other  two  dimen¬ 
sions.  When  attending  to  extrapersonal  visual  space,  for  instance,  our 
attention  is  actually  biased  towards  the  upper  and  right  hemifields  (9).  In 
part,  the  upper  field  bias  for  far  visual  attention  stems  from  the  fact  that 
objects  in  personal  visual  space  generally  appear  in  the  lower  visual  field. 

It  also  relates  to  the  fact  that,  in  most  individuals,  the  brain  areas 
processing  visual  information  from  the  upper  right  quadrant  of  the  visual  field 
possess  specialized  mechanisms  for  perceiving  the  more  detailed  images 
associated  with  far  vision  (9).  This  feature  of  our  visual  system  strongly 
implies  that  the  most  important  alphanumeric  information  on  the  HUD  display 
(’.e.,  altimeter  readings)  should  be  placed  in  its  upper  right  quadrant,  since 
the  pilot  will  ideally  be  attending  to  the  distant  OTW  environment  when  viewing 
the  HUD.  As  discussed  in  a  recent  theoretical  review  (9),  the  three-dimen¬ 
sional  structure  of  visual  attention  presumably  resembles  the  model  shown  in 
Figure  1.  A  prototype  prioritization  of  HUD  visual  space,  based  on  this 
hypothetical  structure  as  well  as  the  relative  importance  of  various  types  of 
flight  information,  is  shown  in  Figure  2.  It  should  be  noted  that  this 
arrangement  is  similar  to  that  of  many  traditional  "T"  displays,  but  differs 
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rigure  1.  Hypothetical  three-dimensional  model  of  visual  attention.  In  most 
individuals,  the  "far"  attentional  system  is  biased  towards  the 
upper  and  right  hemifields,  while  the  "near"  attentional  system  is 
biased  towards  the  lower  and  left  hemifields.  The  most  important 
quadrant  for  far  vision  is  the  upper  right  stippled  area,  while  the 
lower  left  one  is  most  important  for  near  vision. 
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Figure  2.  Prioritization  of  HUD  space  based  on  the  three-dimensional  struc¬ 
ture  of  visual  attention.  Since  far  vision  is  biased  towards  the 
upper  and  right  hemifields  (9),  the  theoretically  most  desirable 
locations  on  the  HUD  during  sustained  out-the-window  focus  are  as 
numbered.  Functions  contained  in  each  quadrant  are  related  to 
their  crosscheck  importance. 
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from  most  current  HUDs  in  that  no  crossing  cf  the  vertical  meridian  is  required 
to  process  altitude  and  airspeed  data. 


A  major  problem  from  the  standpoint  of  display  design  is  how  to  sustain  the 
pilot's  attention  on  the  far  visual  environment  while  still  allowing  him  to 
maintain  proper  spatial  orientation.  Ordinarily,  spatial  orientation  is 
achieved  using  inputs  from  both  the  visual  and  vestibular  systems,  but  pilots 
must  learn  to  depend  more  on  visual  orientation  information  since  vestibular 
inputs  in  the  abnormal  acceleratory  environment  of  flight  are  frequently  non- 
veridical  (10).  The  mode  of  visual  processing  used  in  maintaining  spatial 
orientation  is  referred  to  as  "ambient"  vision  (11),  and  is  generally  believed 
to  take  place  over  the  entire  visual  field,  without  recourse  to  large  atten- 
tional  resources.  Under  normal  circumstances,  we  employ  our  peripheral  visual 
capabilities  to  maintain  proper  orientation  while  reserving  our  focal 
attentional  resources  for  fovea!  vision,  where  contour  and  color  analysis  are 
best  performed.  Unfortunately,  the  relatively  small  HUD  fiel d-of-view 
precludes  the  use  of  peripheral  vision  to  maintain  visual  orientation 
ambiently.  However,  since  one  of  the  brain  areas  predominantly  involved  in 
ambient  vision--the  posterior  parietal  cortex  (12) --is  also  linked  to 
lower-field,  near-vision  functions  (9),  it  may  be  possible  to  devise  altern¬ 
ative  means  of  maintaining  ambient  spatial  orientation  while  the  pilot's  focal 
attentional  resources  are  directed  towards  far  visual  space. 

Thus,  one  of  the  major  challenges  for  a  well -designed  physiological  HUD 
display  would  be  to  allow  the  pilot  to  devote  his  focal  attentional  resources 
to  far  vision  and  to  information  which  requires  focal  processing  (such  as 
digital  altimeter  readings)  while  ensuring  that  spatial  orientation  (i.e, 
attitude  control)  be  performed  in  the  ambient,  near-vision  realm.  Fortunately, 
this  task  is  actually  quite  similar  to  those  performed  during  such  everyday 
activities  as  reaching  for  objects  and  locomotion.  During  reaching,  for 
instance,  the  eyes  are  fixated  on  the  more  distant  object  while  the  hand  is 
almost  effortlessly  guided  to  it  from  the  proximal  lower  visual  field.  It  has 
been  hypothesized  that  global  perceptual  capabilities  in  near  vision  developed 
to  facilitate  this  guidance  (9),  and  it  may  be  possible  to  utilize  such 
specialized  mechanisms  in  maintaining  spatial  orientation  with  the  HUD. 
Likewise,  specialized  lower-field  ambient  mechanisms  may  assist  in 
figure-ground  segregation  and  optical  flow  analysis  during  locomotion  through 
the  visual  environment  (13,14).  These  mechanisms  allow  us  to  trespass  over 
uneven  terrain  and  avoid  obstacles  in  our  path  without  even  being  consciously 
aware  of  them. 

In  summary,  an  analysis  of  the  3-D  structure  of  visual  attention  suggests 
that,  in  addition  to  the  adoption  of  a  quadrant  layout,  two  effective  ways  to 
maintain  spatial  orientation  while  releasing  focal  attention  for  far  vision 
would  be  to  a)  exercise  the  global  perceptual  capabilities  of  our  near-vision 
attentional  system,  and  b)  utilize  those  cues  which  contribute  to  preattentive 
figure-ground  segregation  and  related  functions.  The  following  two  sections 
will  focus  more  directly  on  the  role  of  global  perceptual  and  preattentive 
cueing  in  HUD  symbology  design. 
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GLOBAL  FORM  PERCEPTION 


In  this  section,  an  outline  of  global  form  perception  as  it  is  carried  out 
by  the  human  visual  system  is  presented.  The  importance  of  this  mode  of  pro¬ 
cessing  for  future  HUD  designs  lies  not  only  in  its  association  with  near 
vision  (thereby  freeing  up  focal  perceptual  resources  for  far  vision),  but  also 
in  its  ability  to  create  vivid  percepts  which  are  minimally  dependent  on  actual 
contour  information  (thereby  resulting  in  reduced  display  clutter  and  enhanced 
"see-through"  visibility).  The  general  benefits  of  HUD  symbologies  capable  of 
exploiting  the  global  perceptual  capabilities  of  the  human  visual  system  have 
been  alluded  to  previously  (15). 

Investigations  into  the  nature  of  global  perceptual  processes  date  back  to 
the  Gestalt  theorists  of  the  early  1900' s,  although  the  past  two  decades  have 
witnessed  a  tremendous  expansion  in  our  knowledge  of  them  (16-19).  Briefly, 
"global"  form  perception  differs  from  "local"  form  perception  in  that  it  is 
more  spatially  distributed  and  less  dependent  on  local  contour  detection. 
Because  of  this  difference,  global  processes  can  tolerate  a  much  greater  amount 
of  defocus,  diplopia,  and  other  types  of  visual  degradation.  Much  perceptual 
and  physiological  evidence  links  global  form  perception  to  near  vision,  and  it 
has  been  theorized  that  one  of  the  major  functions  of  global  form  perception  is 
to  perceive  the  movement  of  the  hand  in  the  near  lower  field  while  reaching  for 
more  distant  objects  (9).  In  this  situation,  the  image  of  the  fixated  object 
in  extrapersonal  space  is  focused  and  stationary  while  the  image  of  the  rapidly 
moving  hand  is  defocused  and  degraded.  Although  the  HUD  environment  is 
different  from  an  optical  standpoint  (i.e.,  the  display  itself  is  perceived  as 
much  closer  to  the  optical  distance  of  the  outside  world,  thereby  reducing  the 
amount  of  diplopia  and  defocus),  it  does  resemble  the  reaching  situation  from 
an  attentional  perspective  in  that  focal  attention  must  typically  be  directed 
OTW  while  global  (ambient)  processing  is  free  to  be  utilized  more  proximally  in 
maintaining  spatial  orientation  (attitude)  control.  One  of  the  most 
interesting  lines  of  visual  research  during  the  past  two  decades  has  involved 
the  study  of  illusory  contours,  a  classic  example  of  global  form  perception. 

As  shown  in  Figures  3a  and  3b,  illusory  contours  are  perceived  through 
perceptual  interpolations  and  correlations  rather  than  point-to-point 
correspondences.  As  with  other  global  form  processes,  motion  enhances  these 
percepts  in  a  rather  dramatic  way,  especially  with  large  texture  elements 
(Figure  3b).  This  is  because  global  form  perception  is  largely  mediated  by 
transient,  low-spatial  frequency  channels  in  the  visual  system  (9,20,21), 
presumably  because  of  the  large  and  rapidly  moving  image  of  the  hand  during 
reaching.  As  with  low-frequency  gratings,  global  forms  are  generally  perceived 
more  readily  than  are  local  forms  (17),  despite  the  fact  that  they  are  not 
defined  by  a  continuous  contour. 

The  use  of  global  perceptual  symbologies  may  permit  a  global  construct  of 
the  head-down  attitude  indicator  (AI) --whose  superiority  over  the  current  HUD 
attitude  format  has  been  convincingly  demonstrated  (22-24)--to  be  integrated 
with  the  pitch-ladder  format  of  current  HUDs.  A  prototype  of  such  a  global 
attitude  display  is  shown  in  Figure  4a.  In  addition  to  its  see-through  quality 
and  its  potential  to  exploit  ambient,  near-vision  perceptual  resources,  this 
display  offers  several  advantages  over  the  current  HUD  attitude  format  (Figure 
4b).  The  additional  benefits  include: 
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Figure  3.  Two  types  of  illusory  contours:  (a)  stationary;  (b)  moving. 


Figure  A 
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Comparison  between  (a)  a  HUD  pitch  ladder  in  the  form  of  a  global 
percept,  and  (b)  the  current  HUD  format,  in  both  wings-level  (top) 
and  banked  (bottom)  aircraft  attitudes. 


o  its  similar  shape  to  head-down  Als  and  prototype  flat-panel 
attitude  displays  that  may  eventually  be  placed  near  the  HUD, 

o  its  capability  of  being  perceived  under  degraded  visual  conditions 
(e.g.,  glare,  flashblindness,  cloud  and  ground  masking),  since  global 
perception  and  other  correl ati onal  and  ambient-type  perceptual  processes 
tolerate  more  image  degradation  than  does  local  form  perception 
(9,11,16,18,20,25), 

o  its  overall  stabilization  in  a  set  position  on  the  display, 
eliminating  the  obstruction  of  other  critical  information 
during  slewing  of  the  pitch  scale,  and 

o  its  integration  of  precise  pitch  and  roll  information  in  a  unified  dis¬ 
play,  thereby  freeing  the  pilot  from  the  need  to  constantly  switch  his 
focal  attention  among  the  pitch  ladder,  sky-pointer,  and  bank  scale. 

In  the  context  of  pitch  and  roll  scales,  another  important  and  somewhat 
counter-intuitive  property  of  many  global  percepts--namely ,  their  remarkable 
precision--should  be  noted.  It  has  been  shown,  for  instance,  that  in  vertical 
alignment  and  related  tasks  in  which  the  individual  elements  are  widely  spaced, 
global  perceptual  performance  exhibits  a  precision  which  actually  surpasses  the 
spatial  resolution  of  the  fovea  (25).  Thus,  global  perceptual  cueing,  if 
utilized  properly,  offers  the  possibility  of  eliminating  the  myriad  and 
redundant  use  of  fine  lines  and  digits  on  current  HUD  displays  without  a  con¬ 
comitant  loss  of  precise  aircraft  control. 

In  summary,  the  presentation  of  the  central  attitude  display  as  a  global 
percept  may  represent  a  potentially  major  improvement  over  current  HUD  attitude 
symbology,  at  little  or  no  cost  in  terms  of  display  clutter.  As  with  other 
symbologies,  the  precise  size  and  form  of  the  global  display  may  have  to  vary 
somewhat  during  specific  flight  modes  (e.g.,  weapon  delivery,  landing,  etc.)  to 
achieve  a  maximum  versatility  in  performance.  From  the  standpoint  of  "natural¬ 
ness",  a  global  attitude  indicator  should  be  ideal,  since  the  effectiveness  of 
global  percepts  is  generally  related  to  the  extent  to  which  they  simulate 
"natural"  percepts.  For  example,  a  particular  global  motion  pattern  is  more 
likely  to  be  detected  if  it  has  occurred  in  our  previous  perceptual  experience 
(19).  Indeed,  it  is  doubtful  whether  a  purely  abstract  set  of  computational 
algorithms  could  ever  approach  the  remarkable  global  perceptual  achievements  of 
the  human  visual  system,  even  given  the  impressive  neural  architecture  of 
visual  cortex. 


PREATTENTIVE  FORM  PERCEPTION  AND  ITS  ECOLOGICAL  BASIS 

Powerful  preattentive  perceptual  mechanisms  exist  in  the  human  visual  sys¬ 
tem  that  assist  in  the  segmentation  of  objects  from  the  background  scene  during 
locomotion  and  object  search.  Indeed,  the  ability  to  engage  in  other  endeavors 
(e.g.,  thought  or  conversation)  while  walking,  driving  a  car,  or  searching  for 
objects,  clearly  demonstrates  how  few  attentional  resources  are  necessary  for 
such  behaviors.  Perhaps  even  more  so  than  global  perception,  preattentive  form 
perception  bears  an  intimate  connection  with  the  ecological  principles 
governing  our  interaction  with  the  everyday  visual  environment.  Thus,  it  may 
be  argued  that  a  well-designed  HUD  display  requiring  minimal  focal  attentional 
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resources  will  ensure  that  orientation  control  is  performed  in  a  manner 
consistent  with  the  ecological  basis  of  human  visual  perception. 

The  ecological  view  of  visual  perception  was  emphasized  by  Gibson  (13),  who 
argued  that  the  natural  visual  world  provides  a  rich  source  of  powerful  and 
invariant  information  regarding  the  position  and  movement  of  objects  (incluoing 
ourselves)  through  the  visual  environment.  Included  among  such  natural  cues 
are  perspective,  size  ana  texture  gradients,  kinetic  occlusion,  and  optical 
flow.  It  was  the  original  intent  of  the  AI  to  serve  as  a  porthole  to  the  out¬ 
side  visual  world  (26),  but  the  physical  constraints  of  the  gyro  display  pre¬ 
vented  many  natural  cues  from  being  incorporated.  Although  it  will  be  shown 
later  in  this  section  how  a  totally  veridical  representation  of  the  outside 
world  is  inherently  impossible  for  a  HUD  (or  any  other)  attitude  display,  it 
will  also  be  demonstrated  how  current  computer  graphics  techniques  may  permit 
the  delivery  of  several  cues  that  dramatically  expand  the  pilot's  preattentive 
perceptual  processing. 

Preattentive  cues  generally  include  those  which  a)  define  an  object's  posi¬ 
tion  relative  to  other  elements  of  a  scene  (e.g.,  location,  motion  shear, 
color,  size),  or  b)  distinguish  that  object  from  other  objects  (e.g.,  color, 
size,  shape).  Experiments  in  visual  object  attention  indicate  that  the  selec¬ 
tion  of  objects  on  the  basis  of  their  features  is  performed  rapidly,  preatten- 
tively  and  in  parallel,  whereas  the  combining  of  features  into  precise  forms 
during  a  subsequent  feature-integration  stage  requires  focal  attention  (6).  It 
has  also  been  shown  that  color,  followed  by  size  and  shape,  is  the  most  power¬ 
ful  of  the  object  cues  (27),  although  none  of  these  cues  is  as  effective  as  a 
prior  knowledge  of  the  stimulus'  location.  However,  certain  motion  flow 
discontinuities  (e.g.,  shearing  motion)  may  be  even  more  salient  than  color  in 
segregating  elements  of  a  scene  (28),  especially  under  those  degraded  visual 
conditions  in  which  our  motion  sensitivity  exceeds  our  static  form-detection 
capability  (11). 

Unfortunately,  the  location  cue  has  limited  usefulness  in  sky-ground  demar¬ 
cation  because  the  relative  position  of  the  attitude  display's  horizon  is 
always  changing  as  a  consequence  of  the  movement  of  the  aircraft.  Nor  is  color 
likely  to  become  available  on  HUD  displays  of  the  immediate  future.  Motion 
flow  may  be  useful  in  select  instances,  but  totally  realistic  simulations  of 
terrain  flow  in  attitude  displays  would  compromise  the  ability  of  the  HUD  to 
deliver  precise  pitch  information,  as  will  be  discussed  later.  It  is  impor¬ 
tant,  therefore,  that  the  remaining  cues  { i . e . ,  size  and  shape)  be  effectively 
utilized  for  visual  orientation  control.  Figure  5  shows  how  effectively  size 
cueing  promotes  f i gure-ground  segregation,  when  it  is  defined  both  in  terms  of 
thickness  (Figure  5a)  and  line  length  (Figure  5b). 

Current  HUD  pitch  scales  do  not  utilize  size  differences  for  discriminating 
sky-from-ground,  although  such  cueing  has  been  proposed  by  others  (24). 

Although  shape  is  generally  not  as  good  as  size,  it  can  also  be  beneficial  when 
used  properly.  For  instance,  the  discrimination  of  broken  from  dashed  lines  in 
current  HUD  pitch  scales  can  in  most  cases  be  performed  preattentively, 
although  under  extreme  visual  blur  conditions  (e.g.,  during  high-speed  aircraft 
roll)  individual  segments  in  a  broken  line  may  smear  together  and  lead  to  the 
perception  of  a  solid  line.  On  the  other  hand,  the  discrimination  of  rotated 
L's  from  upright  ones  cannot  be  performea  preattentively  for  180-deg  rotations 
(Figure  6a),  but  can  be  for  90-deg  rotations  (Figure  6b).  Surprisingly,  the 


8 


Figure  5. 


Figure  6. 


Figure  7. 


(•) 


(b) 


Effectiveness  of  size  cues  in  preattentive  form  perception:  (a) 
thickness,  and  (b)  length. 
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(b) 

Comparison  of  elementary  form  segregation  using  (a)  upright  vs. 
rotated,  and  (b)  upright  vs.  inverted  discriminations. 


Illustration  of  two  types  of  ecological  figure-ground  discrimina¬ 
tions:  (a)  stationary  figure/stationary  ground,  and  (b)  moving 
figure/moving  ground,  as  in  forward  locomotion.  Figure  7a  is 
adapted  from  Gibson  (29). 
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discrimination  of  180-degree  inverted  symbols  is  required  by  the  positive  and 
negative  pitch-scale  formats  of  most  current  HUDs. 


It  appears  that  the  effectiveness  of  various  preattentive  cues  may  relate 
to  their  ecological  validity,  particularly  as  they  are  used  in  figure-ground 
segmentation  during  locomotion.  For  example,  nearer  objects  differ  in  both 
size,  shape  (perspecti val  slant),  and  magnitude  and  direction  of  linear  optical 
flow  from  those  in  the  visual  background  (see  Figures  7a  and  7b).  Thus, 
discrimination  of  vertical  vs.  diagonal  lines  aids  in  figure-ground 
segregation,  whereas  upright  vs.  inverted  line  discriminations  do  not.  Like¬ 
wise,  discrimination  of  opposite  rotational  movement  is  much  harder  than 
discrimination  of  motion  shear  (30),  which  more  effectively  simulates  the  rela¬ 
tive  translation  of  objects  during  natural  locomotion.  On  the  other  hand,  cues 
such  as  color  may  be  more  related  to  object  attention  and  may  have  acquired 
their  salience  during  the  emergence  of  fruit-eating  and  fruit-searching  in 
primates  (9).  In  this  context,  it  is  worth  noting  that  recent  theories  of 
primate  vision  have  distinguished  between  the  motion  and  color/form  pathways  in 
the  brain  (9,31).  Accordingly,  the  salience  of  various  preattentive  cues  may 
correspond  to  the  properties  of  neurons  located  in  early  segments  of  these  two 
pathways.  For  instance,  shearing  motion  is  effective  for  stimulating  cells  in 
middle  temporal  (MT)  visual  cortex  (32),  part  of  the  "motion"  system,  whereas 
color  is  a  salient  cue  for  many  cells  in  V4  (33),  linked  to  the  "form"  system. 

The  link  between  preattentive  cueing  and  ecological  perception  implies  that 
a  HUD  attitude  symbology  which  ideally  simulates  the  actual  terrain  movement 
during  flight  will  inevitably  result  in  good  preattentive  cueing.  Thus,  future 
HUD  symbologies  should  utilize  not  only  perspective,  size  and  texture 
gradients,  and  motion  flow,  but  also  such  natural  features  as  mountainous  con¬ 
tours  on  the  horizon  and  simulated  landing  strips  during  descent  portions  of 
the  mission.  The  three-dimensional  richness  of  "natural"  attitude  displays 
(see  Figures  8a-d)  would  even  make  possible  preattentive  (albeit  gross)  estima¬ 
tions  of  altitude  and  airspeed,  and  would  extend  the  HUD's  role  to  specialized 
situations  such  as  the  vertical  descent  during  Harrier  landings  (where  the  rate 
of  optical  expansion,  or  "looming",  could  reflect  altitude  and/or  descent  velo¬ 
city). 

Unfortunately,  the  other  purposes  of  the  HUD  attitude  display  require  that 
certain  compromises  be  made  in  substituting  "HUD  ground"  for  "true  ground". 

For  instance,  size  gradients  are  limited  because  overly  expanded  pitch  lines 
would  result  in  an  unacceptable  occlusion  of  the  display.  Also,  pitch  lines 
must  be  equally  spaced  in  order  to  be  "conformal"  to  the  outside  world,  thereby 
precluding  a  veridical  depiction  of  texture  gradients.  Fortunately,  size  and 
horizontal  spacing  gradients  by  themselves  appear  to  effectively  simulate  the 
natural  gradients  in  our  ecological  visual  representation.  Finally,  the  pitch 
lines  on  the  attitude  display  must  remain  stationary  in  a  constant-pitch  atti¬ 
tude,  so  they  cannot  truly  depict  the  optical  flow  of  the  terrain  underneath 
the  plane.  However,  a  global  percept  capable  of  crudely  simulating  actual 
motion  flow  is  achievable  using  the  technique  of  luminance  "sweeping",  which 
involves  the  progressive  shift  of  a  brief  luminance  increment  across  a 
stationary  two-dimensional  image. 

It  must  be  conceded  that  even  an  ecologically  valid  attitude  display  may 
still  not  address  all  of  the  reputed  "unnatural ness"  of  current  HUD  displays. 
For  example,  information  on  currently  mandated  moving-tape  altitude  and  air- 
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Figure  8.  Global  HUD  attitude  display  employing  a  three-dimensional,  ecol¬ 
ogical  cueing  structure,  in  four  aircraft  orientations:  (a) 
wings-1 ev el  flight/high-altitude, •  (b)  wings-level  flight/low-alti¬ 
tude;  (c)  level  45-deg  turn;  and  (d)  15-deg  nose-down  attitude. 
Arrows  denote  approximate  directions  of  motion  flow. 
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speed  scales  may  be  difficult  to  process,  but  an  understanding  of  ecological 
visual  perception  may  be  of  limited  relevance  in  refining  or  revamping  such 
scales.  It  must  also  be  conceded  that  many  HUD  tasks  cannot  be  performed 
preattenti vely  and  in  parallel,  including  those  requiring  identification  of 
letters  or  digits,  or  fine  orientation  discriminations.  Still,  it  would  be 
advisable  in  these  cases  to  present  only  one  digital  or  fine  analog  readout  per 
HUD  quadrant  to  maximize  the  efficiency  of  the  pilot’s  focal  attentional 
resources.  Also,  displays  which  require  focal  attentional  switching  to  perform 
two  related  tasks  (e.g.,  pitch  and  bank  control )  needlessly  distract  the  pilot 
from  directing  his  focal  attention  in  the  critical  OTW  direction. 

In  summary,  an  analysis  of  the  role  of  preattentive  cueing  indicates  that 
certain  HUD  symbologies  will  allow  the  pilot  to  sustain  his  focal  attention  on 
the  OTW  environment  while  maintaining  good  spatial  orientation  preattenti vely . 
Generally,  the  effectiveness  of  various  preattentive  cues  relates  to  their 
ecological  relevance,  so  that  a  realistic  simulation  of  the  outside  visual 
world  will  almost  assuredly  facilitate  good  preattentive  processing.  Like  the 
global  AI  percept  described  in  the  preceding  section,  preattentive  cueing  in 
the  "physiological  HUD"  will  lead  to  a  futuristic  display  which  nevertheless 
blends  in  a  very  traditional  attitude  display  concept. 


SPATIAL  ORIENTATION  AND  THE  VISUAL  FRAME  OF  REFERENCE 

One  of  the  long-standing  issues  in  attitude  display  design  concerns  the 
proper  visual  frame  of  reference  (34).  Specifically,  should  the  aircraft  be 
portrayed  as  being  stable  while  the  visual  background  moves  (simulating  actual 
retinal  image  motion)  or  should  the  aircraft  be  portrayed  as  moving  against  a 
stationary  visual  background  (as  better  fits  our  perceptual  experience)?  This 
distinction  is  illustrated  in  Figure  9,  as  applied  to  the  case  of  aircraft  roll 
motion.  Although  the  inside-out  perspective  has  been  universally  deployed  in 
modern  aircraft,  it  has  also  been  shown  to  be  the  source  of  much  confusion  in 
both  laboratory  studies  (34,35)  and  early  pilot  training  (Ercoline,  personal 
communication),  thereby  suggesting  that  it  is  much  more  confusing  than 
alternate  reference  frames. 

The  fact  that  the  world  appears  stable  to  us  during  head  and  body  movements 
has  generated  a  great  deal  of  theoretical  speculation  dating  at  least  as  far 
back  as  Helmholtz  in  the  late  1800's.  Most  researchers  have  postulated  that  a 
corollary  discharge  signal  generated  extraretinal ly  instructs  our  perceptual 
system  to  ignore  the  retinal  motion  generated  by  egocentric  movements,  thereby 
stabilizing  our  perceptual  world.  The  source  of  the  extraretinal  signal  is  not 
entirely  known,  but  a  recent  review  concluded  that  at  least  some  of  it  must 
have  a  central  (brain)  origin  (36). 

The  precise  extent  of  extraretinal  stabilization  may  depend  on  the  relative 
weighting  given  to  visual  vs.  other  types  of  inputs  in  perceiving  self-motion. 
Specifically,  the  degree  to  which  retinal  motion  is  relied  on  for  perceiving 
self-motion  seems  to  be  inversely  related  to  the  strength  of  the  corresponding 
vestibular  percept,  the  most  important  extraretinal  source  of  spatial  orien¬ 
tation  information.  Accordingly,  the  HUD  display  should  reflect  this 
relationship  by  using  visual  background  motion  to  code  changes  in  aircraft 
attitude  in  those  situations  normally  emphasizing  retinal  inputs,  and  using 
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Figure  9.  Illustration  of  the  difference  between  a)  an  "inside-out"  frame  of 
reference,  and  b)  a  modified  "inside-out"  display  in  which  roll  is 
depicted  using  a  moving  aircraft  symbol  against  a  stable  horizon. 


aircraft  symbol  motion  in  those  instances  in  which  we  rely  more  on  our  vestib¬ 
ular  sense. 

One  application  of  this  concept  led  to  a  "kinalog"  display  in  which  high- 
frequency  motion  (better  sensed  by  the  vestibular  system)  was  coded  by  moving 
the  object  against  the  background  display,  whereas  low-frequency  motion  better 
sensed  by  the  visual  system)  was  coded  by  movement  of  the  background  scene 
behind  a  stationary  aircraft  symbol  (37).  A  somewhat  less  confusing  situation 
involves  the  use  of  different  frames  of  reference  in  different  axes  of  motion. 
For  example,  visual  scene  motion  much  more  powerfully  induces  a  self-motion 
percept  in  the  yaw  and  pitch  axes  than  in  the  roll  axis  (38),  whereas,  con¬ 
versely,  the  sensitivity  of  the  vestibular  system  is  apparently  greater  in  roll 
than  in  pitch  (39,40).  Indeed,  the  important  role  of  the  visual  field  in 
stabilizing  our  perceptual  world  in  the  horizontal  (yaw)  and  vertical  (pitch) 
dimensions  leads  to  a  certain  ambiguity  concerning  the  motion  of  a  stationary 
object  against  a  moving  background  along  these  axes.  For  example,  fixation 
upon  a  static  dot  surrounded  by  a  moving  background  field  can  lead  to  the 
illusion  that  the  dot  itself  is  moving  in  the  opposite  direction  (41),  as  can 
easily  be  demonstrated  by  observing  the  moon  in  the  midst  of  rapidly  moving 
clouds. 

The  above  analysis  suggests,  therefore,  that  it  may  be  more  important  to 
depict  a  stationary  horizon  and  scene  during  roll  motion  than  during  pitch 
motion.  Such  a  relationship  was  adopted  by  the  Crane  Flitegage  display  (10), 
one  of  the  earliest  attitude  displays;  and  at  least  one  study  has  demonstrated 
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a  greater  advantage  of  outside-in  displays  in  the  roll  vs.  pitch  plane  (35). 

In  addition  to  any  theoretical  or  empirical  justification,  a  major  practical 
consideration  favors  the  use  of  the  "split-frame"  perspective  in  future  HUD 
displays.  This  consideration  involves  the  fact  that  a  20-deg  HUD  display 
cannot  code  the  full  range  of  possible  pitch  or  yaw  motion,  whereas  depiction 
of  360  degrees  of  roll  motion  in  the  frontal  plane  can  occur  without  resorting 
to  moving  the  background  image. 

i 

Despite  the  confusions  associated  with  the  inside-out  frame  of  reference, 
it  may  be  contended  that  an  "outside-in"  display  will  prove  even  more  percep¬ 
tually  disruptive  because  it  is  nonconformal  to  the  retinal  image  of  the  out¬ 
side  visual  world  (i.e.,  the  horizon,  though  perceptually  stable,  does  in  fact 
move  during  aircraft  roll  motion).  If  our  perceptual  system  levels  a  tilted 
horizon  to  give  us  the  perception  of  self-motion  in  a  stable  visual  world,  will 
it  not  also  rotate  a  level  HUD  horizon  by  the  same  amount  and  thereby  produce  a 
perceived  tilt  in  the  attitude  readout?  Although  this  question  requires 
further  research,  subjective  reports  from  pilots  suggest  that  A1  motion  may  be 
perceptually  decoupled  from  motion  of  the  outside  world  during  a  roll.  Hence, 
the  AI's  horizon  may  appear  to  tilt  despite  its  conformance  to  the  actual  hori¬ 
zon,  which  appears  stable.  This  rather  paradoxical  phenomenon  may  relate  to  an 
important  principle  derived  from  laboratory  vection  experiments:  namely,  our 
visual  orientation  system  behaves  differently  for  large  background  vs.  small 
foreground  visual  stimuli  (42,43).  Indeed,  it  has  been  shown  that  an  expand 
sive,  distant  visual  field  exerts  a  strong  influence  over  our  orientation 
sense,  whereas  a  small  central  field  does  not. 

The  reason  for  discounting  motion  in  the  proximal  visual  field  is  probably 
due  to  the  nature  of  our  perceptual  experiences  in  that  realm.  For  instance, 
movement  of  the  limbs  in  near  vision  is  often  independent  of  our  own  bodily 
movements,  while  proximal  images  in  moving  vehicles  are  generally  stabilized 
with  respect  to  the  body.  In  neither  of  these  cases  would  visual  motion  (or 
its  absence)  be  very  useful  in  determining  our  orientation  with  respect  to 
earth.  Thus,  cockpit  display  images,  generally  contained  in  the  central  30 
degrees  of  the  visual  field,  are  not  perceptually  stabilized  in  the  same  way 
that  the  ground  and  horizon  are.  Rather,  such  images  are  more  likely  to  move 
in  the  same  direction  as  the  vestibular  percept,  as  has  been  wel 1 -documented 
for  the  oculogyral  and  oculogravic  illusions  (10). 

In  summary,  an  analysis  of  the  nature  of  visual -vestibular  interactions  in 
humans  leads  to  the  recommendation  that  a  split  frame  of  reference  be  used  on 
the  HUD  attitude  display,  wherein  aircraft  roll  is  depicted  using  an  outside-in 
perspective.  Although  optically  nonconformal  to  the  outside  world,  the  out- 
side-in  perspective  may  ultimately  prove  much  more  consistent  with  the  actual 
visual  perceptual  environment  of  flight. 


CONCLUSIONS 

Conceptual  guidelines  for  a  new  HUD  attitude  symbology  have  been  presented 
and  discussed  in  the  preceding  sections,  based  on  an  understanding  of  the 
physiological  and  ecological  basis  of  human  visual  perception.  Several  design 
recommendations  are  based  on  four  fundamental  aspects  of  visual  perception: 


1A 


o  the  3-D  structure  of  visual  attention. 


o  global  contour  perception, 
o  preattentive  cueing. 

o  visual -vestibular  interactions  in  near  vs.  far  visual  space. 

Although  the  proposed  design  features  incorporate  many  recent  perceptual 
discoveries  and  computer  graphics  capabilities,  they  may  also  be  viewed  as 
somewhat  conservative  in  that  they  build  upon  previous  concepts  and  displays. 
Indeed,  the  basic  "T"  arrangement,  the  round  AI,  the  attempt  to  simulate  the 
outside  visual  world,  and  the  choice  of  a  split  frame  of  reference  all 
represent  traditional  display  concepts  which  are  restored  and  expanded  upon  in 
the  "physiological"  HUD.  A  full  HUD  display  incorporating  the  major  design 
principles  advocated  in  this  paper  would  resemble  the  prototype  shown  in  Figure 
10.*  In  addition  to  its  other  features,  this  design  limits  the  overall  amount 
of  digital  and  analog  readouts  and  eliminates  the  moving-tape  altimeter  and 
airspeed  scales.  An  expanded  version  of  this  display  would  be  more  important 
for  landing  (Figure  11),  whereas  a  decluttered  version  would  be  more  suitable 
in  the  weapon-delivery  mode  (Figure  12).  Collectively,  these  and  similar 
prototypes  from  other  laboratories  (15,23)  offer  the  promise  that  a 
physiologically  based  HUD  symbology--or  at  least  major  elements  of  it — may 
eventually  be  deployed  on  future  USAF  aircraft. 


*  The  USAF  School  of  Aerospace  Medicine  has  produced  a  videotape 
containing  physiological  HUD  prototype  displays  with  full  motion. 
For  information  about  the  availability  of  this  videotape,  contact 
Dr.  Previc. 
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Figure  10.  Full-format,  prototype  "physiological"  HUD  display. 
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Figure  11.  Prototype  physiological  HUD  display  designed  for  landing,  which 
includes  a  simulated  landing  strip  serving  as  a  glideslope 
indicator. 
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Figure  12.  Prototype  physiological  HUD  display  in  a  decluttered  weapons 
delivery  mode. 
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